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Introduction 


The Pell Grant program helps low- and middle-income students earn degrees and certificates from a 
wide variety of postsecondary schools, including community colleges, vocational schools, liberal arts 
colleges, religious colleges, research universities, and many more. Eligible students can use their Pell 
dollars to finance undergraduate studies leading to four-year college degrees, two-year academic and 
occupational degrees, and certificates of varying lengths in a wide variety of subject areas including 
career programs. However, in order for students to use their Pell Grant, the programs must be offered 
on a for-credit basis, be approved by an accreditor and authorized by a state, and provide no less than 
16 credits or 600 clock hours of instruction over the course of 15 weeks. 


Currently, some stakeholders are advocating for relaxing these program eligibility requirements, sug- 
gesting that even shorter programs can provide a needed alternative to existing certificate and degree 
programs. Advocates also suggest these programs offer increased opportunities for meaningful em- 
ployment to students who are uninterested in or unable to complete a longer program. Proponents 
assert that these very short programs, generally known as workforce programs and not necessarily ac- 
credited, are uniquely flexible and responsive to local industry and employer needs and meet the needs 
of students seeking a rapid path to improved workforce opportunities and earnings. 


Existing sources of support for workforce training programs may suffer from inadequate funding.' Simi- 
lar to students enrolled in programs already eligible for the Pell Grant, cost may be a significant barrier 
for low-income individuals seeking to attend very short-term training programs. In response, legislators 
have proposed expanding Pell Grant eligibility to programs as short as eight weeks, including work- 
force programs that are offered on a not-for-credit basis.2 These programs may receive less scrutiny 
from accreditors and other overseers of quality. 


There is no national source of data on short-term programs under consideration for new Pell Grant 
eligibility, and little is known about the number of programs that would be affected by these proposals 
or how well they currently serve students. Existing research into certificate programs suggests positive 
but modest benefits that may last only a few years, but it fails to distinguish short-term programs from 
longer certificate programs that are already eligible for Pell Grants.* 


In order to move beyond the anecdotal and shed light on the promise to students and taxpayers of 
programs that could become newly eligible for Pell Grants under pending legislative proposals, we 
analyzed data on short-term programs in three states that collect and maintain it: Texas, Missouri, and 
lowa.° Combined, these data are some of the most robust available for short-term programs, yet they 
still do not capture a potentially large segment of existing short-term programs that could be made el- 
igible for Pell Grants. Important information gaps remain, including consistent data on program length 
and student outcomes. Furthermore, strengths and limitations unique to each source of data make it 
difficult to make fair comparisons across states. 


Analysis of the state-level data we received shows that very short-term education and training programs 
are diverse in their length and focus. Some programs provide demonstrable value to students in im- 
mediate employment outcomes while others do not. The outcome data available demonstrate highly 
variable returns. 
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Examples of variation include: 


e Short-term programs have a wide variety of lengths. In Texas, for example, the non-credit short- 
term programs we identified are typically 490 contact hours, but required instruction time for 
these programs ranges from 208 to 598 contact hours. 


e These programs are offered in a wide range of fields, and while non-credit programs appear to 
play a major role in this space, we also identified a substantial number of short-term programs of- 
fered by colleges for academic credit. For example, 57 percent of short-term programs identified 
in Texas are offered for credit. 


e Post-enrollment employment outcomes for students in short-term programs also vary widely. In 
lowa, for example, more than half of students attended programs where graduates earned an 
average of about $35,100. However, 44 percent of students attended health-related programs 
where they went on to earn about $17,200. 


This report begins with an overview of the Pell Grant program's current standards, followed by our 
methodology and brief discussion of our findings. Turning to each state, we proceed to ask what each 
data source shows about the short-term program landscape, including: 


e How many short-term programs currently exist, and how long are they? 
e¢ How much do these programs cost, and how do students currently pay for them? 
e What kinds of industries do they aim to prepare students for? 


e Who currently enrolls in short-term programs, and how well do they fare after enrolling? 


We highlight throughout the key methodological challenges we encountered, which underscore the 
need for better data and data systems capable of assessing the quality and payoff of programs not cur- 
rently eligible for the Pell Grant. 


Current Federal Aid Eligibility, Existing Proposals, and a History 
Meriting Caution 


Each year, the Federal Pell Grant allows 7.5 million students to pursue postsecondary education. Stu- 
dents, schools, and programs must each meet eligibility requirements. Students must demonstrate sig- 
nificant financial need through the federal financial aid application and meet a small number of other, 
limited requirements. The vast majority of Pell recipients have incomes below $40,000. 


Schools must be accredited, authorized by at least one state, and meet other requirements established 
by federal law. However, a school’s eligibility does not necessarily extend to all of its programs. Pro- 
grams must be included within the scope of the school’s accreditation and state authorization. In addi- 
tion, among other requirements, programs must be offered for a minimum of 16 semester credits or 600 
clock hours over the course of 15 weeks to be eligible for Pell Grants. 


Students enrolled in programs too short to qualify for Pell Grants may borrow federal loans, as long as 
those programs provide at least 300 clock hours of instruction over no less than 10 weeks, have been 
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in existence for at least one year, and have verified graduation and job placement rates of 70 percent, 
among other standards.® These requirements were established under the George H. W. Bush Adminis- 
tration in 1992 after high-profile cases of fraud and abuse of federal aid funds and years of high student 
loan default rates.’ While other provisions in the 1992 Amendments to the Higher Education Act were 
aimed at all programs offered by for-profit institutions, Congress applied these requirements to all 
short-term vocational programs, regardless of the institution offering them. However, it is not clear that 
many institutions offer loans to students enrolling in shorter programs. 


Several existing legislative proposals would extend the Pell Grant to programs as short as 150 clock 
hours over as few as eight weeks, including non-credit programs that meet the reduced length require- 
ments. None set any specific standards for completion or labor market outcomes, though some do 
include additional requirements that programs be aligned with guidelines established under the Work- 
force Investment Opportunities Act (WIOA). 


Our analysis focuses on programs too short under current Pell Grant eligibility rules that provide at least 
150 clock hours of instruction over no less than eight weeks. Importantly, because this report focuses on 
what existing data show about these short-term programs already being offered, we cannot take into 
account how the short-term program landscape may change in response to an influx of newly available 
federal funding. New providers may enter the space, and existing providers could create or alter pro- 
grams to make them Pell eligible under new rules. The history of abuse of federal student aid, including 
the altering of program length to evade accountability, underscores the need to carefully approach 
potential changes to current program eligibility standards.® 


Methodology and Data 


We received data on programs in Texas, Missouri, and lowa, each of which we identified from research 
and conversations with experts as having relatively robust postsecondary data systems that include at 
least some information on short-term programs specifically.’ 


Each of our sources of data include a different universe of programs: 
e Texas data include programs in all of the state's community and technical colleges; 


e Missouri data include all education and training programs registered with the state’s Department 
of Higher Education, the Division of Workforce Development, or both; and 


e lowa data include only those programs in the state’s community college system. 


We limited our interest to short-term programs offered either for credit or on a non-credit basis by 

an accredited institution eligible for federal financial aid. We sought information about the programs 
themselves (for example, program length, industry focus, cost), their students (earnings before enroll- 
ment, demographic data), and student outcomes (completion, attainment of industry-recognized cre- 
dentials, post-enrollment employment and earnings). Each data source included different variables, and 
where similar variables were tracked, metrics were calculated differently across different sources. This 

is particularly true for outcome data, which was not widely available. For a complete table of outcome 
information available by source, see Appendix A. 
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Each of our three data sources has strengths and weaknesses.'° For example, Missouri data include 
programs offered by all institutions in the state regardless of whether the institution is public, private, 
or for-profit, and has robust data on direct program costs. However, Missouri was unable to provide 
any student demographic or enrollment data. And while Missouri does calculate wage and employ- 
ment outcomes for the mostly public programs with participant and completion data, the state’s data 
processing systems were not yet at the stage to facilitate inclusion of these measures in the data we 
received. Both lowa and Texas data include state-calculated labor market outcomes. No demographic, 
completion or credential attainment rate data were available from any of the three data states."' 


TABLE 1: Programs and Notable Variables Included in Each Data Source 
INfoltle)(-MVElateloltass 


Student Information (@)Ujixere)ant-Wlancelaaatiieyal 


iaceXele-laa-mlatellele(-Ye| 


Program Cost 


Demographics 
Pre-Enrollment 
Earnings/Employment 
Prior Credit Award 
Attainment 
Completion Rate 
Industry-Recognized 
Credential 
Attainment Rate 

Post-Enrollment 
Earnings/Employment 


Certificate programs in all of the * 
Texas state's community and technical x x x x x x Vv 


colleges (credit and non-credit) 


All programs offered by a Title IV 


Missouri eligible provider ~ x x x x x x 


(credit and non-credit) 


All programs offered in the lowa 


lowa Community College System x x S SA x xX AS 


(credit and non-credit) 


*Available for only a subset of programs 
**Available for too few programs to be included in the analysis 
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Challenges of Identifying Program Length 


None of our three data sources currently tracks short-term programs in a way that allows 
a simple determination of whether their length falls within the range proposed by the leg- 
islative proposals. Program length determinations for Pell Grant eligibility hinge on both 
a minimum amount of instruction time (either credit hour or clock hours) and a minimum 
period of time over which that instruction is received (typically weeks). Both thresholds 
stem from the need for objective standards in program substance, and measures for both 
are well established for traditional for-credit programs. 


However, data on both hours and weeks are not routinely available. All our data sources 
include programs of varying lengths, requiring us to identify the actual program length in 
order to isolate only short-term programs falling within the length of interest. At the same 
time, none of the data we received included both contact or credit hours and the number 
of weeks over which those instruction hours are delivered. For example, data from both 
Texas and lowa include information on a program's credit or contact hours but not the 
number of weeks of a program. Required program length in Missouri data are reported in 
either credits, contact hours, days, or months. In lowa, the contact hour data for non-credit 
programs represent contact hours accumulated, rather than required, for program com- 
pletion, and these data are furthermore reported differently across different reports (for 
example, with precise hours in one file and grouped into three categories in another). For 
more information on the program length data available by source, see Appendix B. 


Moreover, because the existing short-term programs are not currently governed by clear 
standards regarding minimum time over which instruction is received, the same short-term 
program might be offered at the same school with similar hours of instruction delivered 
over different periods of time. A student in a single program requiring 150 contact hours 
could be attending a ten-hour session held once a week for 15 weeks, a ten-hour session 
that runs Monday through Friday for three weeks, or an eight-hour session every Saturday 
and Sunday for just over nine weeks. Short-term programs’ unique potential strength of 
flexible scheduling may itself play a role in complicating program-length eligibility deter- 
minations that align with current standards. 


The federal student aid program furthermore measures instructional time in either cred- 
it hours or “clock hours,” which are defined by regulation.'* However, our data sources 
use either credit hours or “contact hours,” which could mean different things in different 
states. For the purposes of our analysis, we assume the terms “clock hour” and “contact 
hour” are equivalent. 


PAGE 8 SHORT-TERM PROGRAMS IN THE SHADOWS: WHAT DATA SHOW ABOUT PROGRAM LENGTH, COST, AND PAYOFF 


Discussion of Key Findings 


There is no national source of data on short-term programs under consideration for new Pell Grant 
eligibility. Even among the states that collect useful data on such programs, gaps in the available infor- 
mation on duration, providers, fields of study, industry alignment, and student outcomes remain. Policy- 
makers, advocates, and schools exploring an expansion of the federal Pell Grant program to types of in- 
structional activity that remain beyond the scope of current federal data systems needed to administer 
the program will need to address this challenge head on. 


The best available data show that short-term programs vary tremendously in occupational focus, dura- 
tion, enrollment, and student outcomes. Some short-term programs provide value to students, at least 
in terms of short-term employment outcomes. However, results are highly diverse, and some programs 
achieve poor results. 


How Many Short-Term Programs Are There? 


The data we received offer more detail on short-term programs than is available from any federal data 
source. Yet, the true number of short-term programs offered in the systems and states for which we 
have data remains out of reach because required program length data for programs included in estab- 
lished data systems are reported inconsistently across (and sometimes within) different sources. 


Because of variations in state-level data reporting requirements and data systems, the 
number of short-term programs identified in each state are not directly comparable. 
Moreover, short-term program counts may not be complete within a state or system. 
For example, we identified 343 short-term programs in Texas, but an unknown number 
of additional short-term workforce training programs designed to fulfill the customized 
needs of a specific local employer or industry are not overseen by the state and do not 


Short-term pro- 
grams can operate 
without any state 
or system-wide 
reporting require- 


appear in the Texas data. Similarly, programs leading to Occupational Skills Awards, ments, leading to a 
which are a set of sequenced workforce courses designed to meet the WIOA require- potentially signifi- 
ments of the Texas Workforce Commission, may appear in the state’s public labor mar- cant but unknown 
ket outcomes reports but are not captured in the data set that tracks program length.'? number of short- 


term programs not 


What Do Short-Term Programs Look Like? 


Our research suggests a very diverse landscape of short-term programs. These programs are offered 
in a wide range of fields. The data suggest that the focus of short-term programs may vary by state. 
Health-related professions are the most common short-term program focus in Missouri, and in Texas, a 
plurality of short-term programs are associated with education, training, and library occupations. 


At the same time, inconsistencies in how data are reported and the different universes of programs 
included in each data source complicate direct comparisons of program focus across sources. Both 
Standard Occupational Classification (SOC) and Classification of Instructional Programs (CIP) codes can 
be used to shed light on types of short-term program activity. Where SOC codes describe and classi- 
fy occupational categories and can identify the career or job for which a program is designed to train 

a student,'* CIP codes describe categories of educational activity.'° None of our three data sources 
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included each program's associated SOC code, but all had a program's CIP code, which can be linked 
to one or more SOC codes to shed light on which occupational groups are associated with a program's 
educational focus." 


While non-credit programs appear to play a major role in this space, we also identified a substantial 
number of short-term programs offered for academic credit.'’? While all of our data sources include at 
least some non-credit programs, the true share of short-term programs that are non-credit remains 
unknowable using data provided by two of our three states. In Missouri, whose data are inclusive of all 
programs (both credit and non-credit) offered in the state, 32 percent of short-term programs were of- 
fered for credit. Where data include both for-credit and non-credit short-term programs, the breakdown 
of programs by credit status varies tremendously. 


How Long Are Short-Term Programs? 


Available data suggest that the length of programs falling within the range proposed for new Pell Grant 
eligibility can vary widely, particularly among non-credit programs. For example, while the non-credit 
short-term programs identified in the Texas data are typically 490 contact hours, required instruction 
time for these programs ranges from 208 to 598 contact hours. 
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Short-Term Program Structure and Flexible Scheduling 


In Missouri, non-traditional program schedules are common: 61 percent of short-term 
programs identified in the Missouri data are offered during the evening and 31 percent are 
offered on the weekend. Scheduling data were not provided by the other states. Schools 
can and do offer programs that train students for the same credential over different periods 
of time. Other insight about program structure, such as whether or not the program is 

part of a pathway or offered as a stackable credential, remains out of reach in the data 

we received. While such program structure may be common, single programs offering 

a sequence of certifications, or which are part of a broader combination of potential 
certifications, can also present challenges to data reporting. In the same way that a program 
offering training in Microsoft Office Suite may provide students an opportunity to receive 
certifications in each product within the Suite, a program training students for an industry 
recognized certification may include bundled credentials that allow or require a student to 
accumulate multiple certifications along the way to completing a single program. Bundled 
credentialing can complicate the interpretation of reported outcomes, especially where 
bundled credentials are not clearly distinguishable by award size in the same way credit 
awards can be counted by “highest award.” 


Furthermore, depending on the credential or credentials for which a program trains 
students, a distinct program with a distinct credential offered at one school can be offered 
as part of a longer pathway or bundled with other credentials at another school. This 
complicates fair comparisons across different institutions. 


Program-level data may mask what is part of an individualized, cafeteria style, or guided 
pathway structure that offers students multiple options to the same credential or multiple 
but connected programs to continue training or upskilling. Where data reporting requires 
a static definition of a program, tying an identifiable industry-recognized credential to a 
program may be one option for ensuring quality. 


Similarly, a single program can lead to a standalone credential, be a sequential step in a 
specific industry pathway, or both. For example, a student may complete a Certified Medical 
Assistant credential and never pursue further training, and another may continue training 
with a Phlebotomy credential.'® Such flexibility in the short-term program space, which 

may be a unique strength of short-term programs, makes it difficult to distinguish in the 
data we received how different programs and credentials may relate to each other. It also 
makes it difficult to understand how the path on which students pursue various credential 
opportunities relates to their economic outcomes following credential attainment. 
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How Are Programs Currently Funded? 


Our data do not clarify what federal, state, institutional, and local funding short-term programs current- 
ly receive. Particularly unfortunate are the absence of data on program-specific levels of funding, from 
either employers or through Workforce Innovation and Opportunity Act (WIOA). 


The goal of short-term programs aligns with the purpose of WIOA, which seeks to “help job seekers 
access employment, education, training, and support services to succeed in the labor market and to 
match employers with the skilled workers they need to compete in the global economy.”'? The Center 
for Law and Social Policy (CLASP) notes that Title | of WIOA is “the primary source of federal workforce 
development funding to prepare low-income adults, youth, and dislocated workers for employment 
and to help them continue to build skills once they are employed.””° Existing legislative proposals to 
expand Pell Grants to short-term programs may refer to WIOA standards and encourage alignment 
with existing data reporting and quality assurance mechanisms provided by WIOA. 


Only data from Missouri include an indication of which short-term programs have requested WIOA 
eligibility. These data show that 89 percent of the 199 short-term programs identified in the state are 
registered on the state's eligible training provider list.*' 


A number of programs are also directly supported through donations of equipment from local indus- 
tries and employers, but the number of these programs or employers directly benefiting from these 
arrangements could not be included in any of the data we received. 


Who Enrolls in Short-Term Programs? 


None of our three data sources included information on the demographics of students enrolling in 
short-term programs. lowa is the only data source that captures pre-enrollment educational attainment 
or employment of students in short-term programs. Nine percent of students enrolled in non-credit 
short-term programs in the lowa Community College System had earned a credit award prior to enroll- 
ment. A majority of students enrolled in these programs are employed both prior to and during enroll- 
ment. 


How Do Students Typically Cover Tuition Costs? 


Missouri data offer tremendous insights into the direct costs of short-term programs. However, none of 
our data sources offered any information about the typical amounts students actually pay out of pocket 
or the amount covered by any particular source, including federal or private loans. The lack of data on 
borrowing is a particular shortcoming given that students in some of the programs identified could be 
eligible for federal student loans. 


How Do Students Fare After Short-Term Program Enrollment? 
Information on how students attending short-term programs typically fare are not consistently available 


across our three data sources. The available data suggest that some programs produce good post-en- 
rollment employment and earnings outcomes. Others do not. 
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Earning the industry-recognized credential associated with a single program can involve requirements 
beyond program completion, such as a test or a required number of hours of practice. Short-term 
programs’ promise of a fast path to employment in a specific industry makes credential attainment rates 
particularly relevant to assessing program quality, but no such data were available. 


Likewise, post-enrollment and earnings measures are particularly critical for assessing claims about the 
labor market value offered by short-term programs. Where labor market outcome data are available, 
they tell an inconsistent story. In lowa, for example, the 90 percent of students who were employed fol- 
lowing enrollment in non-credit short-term programs typically earned $27,000 a year after leaving their 
program. At the same time, health-related programs account for 44 percent of non-credit short-term 
program enrollment, and employed students leaving these programs typically earned about $17,200, 
while the remaining 56 percent of employed students earned about $35,100. 


Challenges of Identifying Outcomes Specific to Short-Term Programs, Part 1 


Sources reporting outcome metrics did so in ways that required us to use additional data 
provided to isolate which of the outcomes reflected only short-term programs. Outcomes 
were generally able to be isolated for only a very small share of programs identified in the 
sources for which any outcome data were available. For example, Missouri credential attain- 
ment rate data were reported for all programs sharing a four-digit CIP code, which grouped 
together six-digit CIP programs of different lengths. Credential attainment rates were able 
to be isolated for only 10 short-term programs. Due to this very small sample size, we do 
not report these data. Texas labor market outcome data similarly reflect groups of programs 
of different lengths, and just 7 percent of these program groups included only short-term 
programs. 


The fact that labor market outcomes specific to short-term programs, when available, can reflect only a 
small share of short-term programs identified, calls for caution in generalizing outcomes of these pro- 
grams to the full landscape of short-term programs. 


Only data from Texas and lowa include any labor market outcome data, and the metrics available are 
not consistently defined across sources. For example, Texas’ employment rate is calculated as the share 
of graduates found working during the fourth quarter of the calendar year in which a student complet- 
ed the program, and lowa Community Colleges’ employment rate is calculated as the share of students 
no longer enrolled and found working in at least one of the four quarters following the student's pro- 
gram exit. Small cohort sizes for non-credit programs furthermore lead to the suppression of outcomes 
for a significant number of short-term programs; for example, earnings and employment data are not 
reported for a third of the short-term programs in the lowa non-credit outcome report and 28 percent 
of the short-term program groups in Texas, as a result of small cohort size. 
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The available employment and earnings data show that some short-term programs 


Some short-term can produce labor market benefits for graduates, but typical post-enroll ment earnings 
programs can pay vary widely. Variable labor market outcomes are not unique to the short-term program 
off for graduates, space.”* However, they do underscore the need for quality assurance should additional 
but available labor programs be made eligible for the Pell Grant in order to prevent the proliferation of 
market outcome low-quality programs. At the same time, available data are not able to identify con- 
data tell an sistent characteristics of high-quality short-term programs that are needed to inform 


inconsistent story. — the development of quality guardrails appropriate for ensuring short-term programs 
achieve their specific goals. 


For a full table of the outcome data available by source, see Appendix Table A. 


To What Extent Are Employers Engaged with Program Development? 


Recognizing that short-term programs may offer unique value in allowing nimble responsiveness to 
changing local labor markets and specific local employer needs, legislative proposals seek to require 
some degree of engagement with local industry. 


Employers themselves may play a key role in supporting short-term programs, including through en- 
gagement with curriculum development as well as financial support. Yet our data cannot speak to the 
extent to which local industries or employers support either the development or funding of specific 
programs nor whether providing federal funds could supplant rather than supplement these local in- 
vestments. 


Short-Term Programs in Texas 
About the Data 


Our Texas data consist of a program inventory provided by Texas Higher Education Coordinating 
Board,” and the Texas state Gainful Employment (GE) outcome data available online. Both of these 
data include non-credit (continuing education (CE))* and for-credit (non-CE) Level 1 certificate pro- 
grams offered at Texas public community, state, and technical colleges.”° The program inventory data 
were merged with the 2016 GE data file to identify outcomes specific to short-term programs isolated in 
the inventory file. 


Program Characteristics 


College/Division 


Texas inventory data show 343 short-term programs offered at 70 institutions, over half (57%) of which 
are offered for credit. 
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Program Length 


The 149 identified short-term non-credit continuing education programs in Texas are typically offered 
for 490 contact hours, ranging from 208 to 598 contact hours. All but one of the 194 short-term for-cred- 


it programs in Texas are offered for the equivalent of 563 contact hours (15 credits). 
Industry Focus 


One-third (34%) of the short-term programs in the Texas program inventory are asso- 
ciated with education, training, and library occupations, according to an analysis of 
the SOC codes associated with the CIP codes included in the Texas data. Another 21 
percent are associated with management occupations.” 


The table on the following page shows Texas program length information by occu- 
pational focus, with for-credit programs’ credit hours converted to contact hours.” 
For-credit programs are typically longer than non-credit programs, and their program 
length across occupational focus does not vary anywhere near as much as non-credit 
programs. While for-credit programs are all typically the same length (the equivalent of 
563 contact hours), non-credit Texas programs range from an average of 336 contact 
hours among arts, design, entertainment, and sports programs to 534 among protective 
service programs. Program length also varies widely within non-credit programs sharing 
an occupational focus. For example, the 46 non-credit installation, maintenance, and 
repair programs range from 296 to 576 contact hours. 


Short-term 
programs offered 
for credit in Texas 

are typically 
longer than non- 
credit programs, 
with less variation 
across and within 
industry focus. 
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TABLE 2: Texas Program Length by Industry Focus and Credit Status 


For-Credit Programs 


Sty Non-Credit Programs (Credit Hours** Converted 


Industry Focus (SOC)* Program ixoM @Xolayr-Loiel wColU] ¢)) 


Count 


# 
acexe]c-lants 


Education, Training, & Library aig 42 208 598 495 He 450 563 561 

Management 72 16 336 592 463 56 450 563 560 
Business and Finance Operations 52 12 336 576 456 40 563 563 563 
Production 49 28 368 579 483 21 563 563 563 
Architecture and Engineering 46 18 336 560 449 28 563 563 563 
Installation, Maintenance, and Repair 46 20 296 576 460 26 563 563 563 
Computer and Mathematical 34 10 384 576 482 24 450 563 558 
Healthcare Support 29 23 368 584 496 6 563 563 563 
Protective Service 235} W 400 576 534 8 563 563 563 
Healthcare Practice and Technology 21 12 354 598 470 oO) 563 563 563 
Office and Administrative Support 19 5 368 592 445 14 563 563 563 
Construction and Extraction 16 14 296 576 427 7 563 563 563 
Arts, Design, Entertainment, and Sports 9 2 208 464 336 7 563 563 563 
Personal Care and Service 8 2 464 500 482 6 563 563 563 
Food Preparation and Serving Related if 3 384 432 411 4 563 563 563 
Life, Physical, and Social Science 6 2 376 592 484 4 563 563 563 
Sales and Related 5 0 N/A N/A N/A 5 563 563 563 
Farming, Fishing, and Forestry 3 2 366 376 371 1 563 563 563 
Community and Social Service 2 2 400 573 487 0 N/A N/A N/A 
Building Grounds Cleaning & Maintenance 2 7 398 448 423 0 N/A N/A N/A 
Transportation and Material Moving 2 1 366 366 366 1 563 563 563 
Legal dl 0 N/A N/A N/A 1 563 563 563 


*Texas programs were matched to 2-digit SOC occupational codes using a crosswalk between 2-digit “major” SOC codes and 6-digit CIP codes. A single 
CIP can match to multiple SOC codes, so the same program may be included in more than one industry focus category. 
**Figures are rounded to the nearest 1. 
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Program Outcomes 


Labor Market Outcomes 


Outcome data specific to short-term programs in Texas were limited to 29 groups of similar short- 
term programs at a single institution.”? The outcomes identified for the 29 short-term program groups 
reflect a total of 1,433 graduates (1,300 non-credit graduates, and 133 for-credit graduates). Employ- 
ment counts were available for 19 of 29 short-term program groups, allowing us to derive an overall 
post-completion employment rate of 79 percent.°*° 


Challenges of Identifying Outcomes Specific to Short-Term Programs, Part 2 


Our Texas data come from two different sources that we merge in order to isolate outcomes of short-term 
programs specifically. Unfortunately, the way programs are reported in each of these sources means that 
we were able to identify outcomes for only a very small share of short-term programs. 


Both the Texas program inventory and the state's Gainful Employment (GE) outcome data include a 
six-digit CIP code, credit status, and institution name for each program.*' However, only the inventory file 
includes the program’s length. In other words, the GE file reports outcomes for all certificate programs 
sharing a six-digit CIP and credit status at a single institution regardless of their length (“program groups”). 


For example, Texas’ program inventory report shows two for-credit Administrative Assistant and Secretari- 
al Science, General Certificate programs (CIP 520401) at Coastal Bend College, one of which is classified 
as a short-term program (15 semester credit hours) and one that is not (39 semester credit hours). Mean- 
while, the GE report includes only one for-credit certificate offering under CIP 520401 at Coastal Bend 
College. 


Similarly, the program inventory report shows two non-credit Culinary Arts/Chef Training Certificate pro- 
gram (CIP 120503) at Alvin Community College — one of which is a short-term program (416 contact hours) 
and another that is not (768 contact hours). The GE report shows only one. 


Because labor market outcomes in the GE report are reported for program groups that include programs 
of varying lengths, rather than by programs of unique length, isolating outcomes for the 343 short-term 
programs identified in the inventory data required us to group together programs in the inventory file to 
align with how the data are reported in the GE file.*? 


After creating program groups that allowed us to match the inventory programs to the GE file, we subse- 
quently identified which of those groups contained only short-term programs in order to isolate outcomes 
that reflect only short-term programs. Most (85%) of the program groups created from the inventory file 
contained no short-term programs and 7 percent of the groups included a mix of programs with variable 
lengths. Only program groups that consisted exclusively of short-term programs were included in our out- 
comes analysis.°° 


Ultimately, just 7 percent (140) of the program groups in the inventory file consist exclusively of short- 
term programs, and outcomes were available for just 29 (21%) of those 140 groups, because most of the 
program groups in the inventory were not found in the corresponding GE file, and other outcomes were 
privacy-suppressed due to small cohort sizes. 
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Wage data specific to short-term programs in Texas were even more elusive than employment rates, 
and reflect even fewer programs and students: The $27,500 median annual wage for short-term pro- 
grams reflects just 17 program groups and 1,133 working graduates.™ 


GRAPH 1: Labor Market Outcomes of Texas Programs 


Median 
Annual Wage 
$27,500 


79% 
29 Programs Employed 


1,433 Graduates 


17 programs 
1,133 working 
graduates 


19 programs 
1,139 working 
graduates 


Job Trajectories and Goals of Short-Term Programs 


Short-term programs can be aimed at a number of different goals such as helping an em- 
ployed student increase earnings in the same career, moving an unemployed student into 
the workforce, or facilitating a career switch. None of the data we received can identify each 
program’s particular career trajectory goal. This lack of context, coupled with limited data 
on pre-enrollment employment and earnings, makes assessing any available earnings out- 
comes of short-term program graduates difficult. For example, high post-enrollment earn- 
ings may be a function of already skilled students enrolling to further advance their career, 
and low post-enrollment earnings may reflect a successful outcome for students with few 
other options. 
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The 1,056 graduates of the non-credit programs earned $28,890 and the 77 graduates of the for-credit 
programs earned $25,940. Wages for both non-credit and credit graduates furthermore vary widely, 
from $9,540 (Fire Science/Firefighting program at Tyler Junior College) to $49,920 (Criminal Justice/Po- 
lice Science program at Alamo Community College-San Antonio College) among for-credit programs, 
and from $16,650 (Culinary Arts/Chef Training program at Texarkana College) to $43,710 (EMT Para- 
medic program at Lamar Institute of Technology) among non-credit programs. 


Short-Term Programs in Missouri 
About the Data 


Our Missouri data come from the state's single inventory of all programs registered with its Depart- 
ment of Higher Education, the Division of Workforce Development, or both, including both credit and 
non-credit programs. The $5 million data collection and reporting effort to bridge the gap between 
data on both kinds of programs was supported by a 2014 federal grant that ended in September 2018.°%° 


The state's data collection effort also resulted in the creation of MoSCORES, a searchable online in- 
ventory of all credit and non-credit programs offered at an institution of higher education or an eligible 
training provider in the state.°° On MoSCORES, users can find and compare individual programs, or 
download employment and wage information reports for a selection of programs. Our Missouri data 
are an inventory of all programs on this website, as of December 2018. We received many of the metrics 
displayed on the website, including program cost. Other information displayed for some programs on 
the website, such as state-calculated completion and earnings metrics, as well as demographic informa- 
tion, were not available for enough programs such that those data could be included in what we re- 
ceived.” Data from Missouri did include some credential attainment rate data, but it was only possible 
to match credential rates to 10 of the 199 short-term programs identified in the inventory. 


Program Characteristics 


College/Division 


We identified 199 programs in Missouri that were offered by an institution eligible for federal financial 
aid.*® Sixty-nine percent of these programs are offered at a public two-year college and another 22 
percent (44) are offered at a public four-year college, while 6 percent (12) are offered at a non-profit in- 
stitution and 3 percent (six) at for-profit colleges. About two-thirds (68%) of the short-term programs are 
non-credit and 32 percent are offered for-credit. 


Program Length 


Missouri program data include one measure of program length, expressed in either number of contact 
hours, credit hours, days, weeks, or months.*? Two-thirds (117) of the 199 short-term programs identified 
are non-credit programs with program length measured in contact hours; these programs are typically 
offered for 210 contact hours, ranging from 150 to 588. About a quarter (23% or 41 programs) of the 
short-term programs are for-credit programs with length measured in credit hours. These programs are 
typically 12 credit hours, ranging from nine to 15 credit hours. 
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Eleven percent (19) of short-term programs have a program length measured in weeks. These pro- 
grams, all of which are non-credit, are typically 10 weeks long and range from eight to 14 weeks. 


TABLE 3: Missouri Program Length‘! 


INfolats @ig-Yol ial macele] e-Vaas For-Credit Programs 


Number of Programs We 41 


338-563 contact hours 
Range 150-588 contact hours G5 credits) 


; 338-563 contact hours 
Median 210 contact hours (9-15 credits) 


Industry Focus 
One-quarter of 


the short-term CIP codes in the Missouri data indicate that 30 percent of the short-term programs 
programs in identified in Missouri data are health professions. Another 17 percent of Missouri pro- 
Missouri lead to grams focus on business, management, marketing, and related support. 

state licensure, 

57 percent lead Missouri data also include indications of whether the program leads to state licensure, 
to an industry- federal licensure, or an industry-recognized credential. However, they include no infor- 
recognized mation on the specific associated license or credential. These data show that one-quar- 
credential, and 4 __ ter of the short-term programs identified in the Missouri data lead to state licensure, 
percent lead to 57 percent lead to an industry-recognized credential, and 4 percent lead to federal 
federal licensure. licensure.”* Four programs lead to all three, and 49 programs do not lead to either state 


licensure, federal licensure, or an industry-recognized credential.** 
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Program Cost 


The typical short-term program in Missouri costs students $3,000 (including in-state tuition, fees, books, 
tools, supplies, and other costs paid to the institution).™ Typical direct costs vary widely by the type of 
institution offering the program, from $2,550 at for-profit colleges (six programs) and $2,950 at public 
colleges (161 programs), to $5,020 at non-profit colleges (ten programs). Total Missouri direct program 
cost also varies by program focus, from $1,770 (for ten engineering technologies and engineering-relat- 
ed fields programs) to $3,910 (four homeland security, law enforcement, firefighting, and related Protec- 
tive Services programs). 


Missouri program cost data also include breakouts for different types of direct costs. Books typically ac- 
count for another 5 percent of total program cost (ranging from zero to 41%), and fees 4 percent (rang- 
ing from zero to 29%). Across all short-term programs, supplies typically make up 2 percent of program 
cost (ranging from zero to 9%). The typical Missouri program’s tuition cost makes up 86 percent of the 
total direct program cost, ranging from a low of 40 percent for three types of programs to a high of 100 
percent for another three program areas. 


Program Outcomes 


Our Missouri data do not include any completion rates or labor market outcomes. 


Short-Term Programs in lowa 
About the Data 


Similar to Missouri, lowa has prioritized improving the quality and transparency of data on non-cred- 

it programs and outcomes through a partnership between the lowa Department of Education and 

lowa Workforce Development.* The lowa data we use come almost exclusively from publicly available 
reports on programs offered by the state's community college system, and they include outcomes for 
both credit and non-credit programs.” Data reflect one-year outcomes for the 2015-2016 academic year 
cohort. 


Program Characteristics 


College/Division 


The precise number of non-credit short-term programs in the state's community college system remains 
out of reach with the available data. (See Challenges of Identifying a Unique Short-Term Program on the 
following page.) 


Current Pell eligibility determinations for for-credit programs were provided by lowa Community Col- 
lege System, based on both credit hours and linkages to other programs. Due to the way lowa has 
structured many of its for-credit short-term programs as part of a larger program, 145 programs with 
fewer than 16 credit hours are already eligible to receive Pell Grants. In fact, just four for-credit short- 
term programs are not currently eligible for the Pell Grant. 
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Challenges of Identifying a Unique Short-Term Program 


Both credit and non-credit programs are classified with CIP codes, which provide a shared 
structure for tracking and reporting programs by field of study. While a CIP code can effec- 
tively categorize programs by substantive category, programs at the same institution report- 
ed under the same CIP code can still vary in award level, instruction time, length, and credit 
status. These variations make a CIP code alone insufficient to identify unique programs at a 
single college. 


Contact hours can be helpful in distinguishing non-credit short-term programs sharing a 
CIP from longer programs, but existing data systems may not track required contact hours. 
Where contact hour data are available, they can represent accumulated, as opposed to 
expected or required, enrollment time. And they can furthermore be reported differently 
depending on the purpose of a report. 


Moreover, an effort to derive a school’s non-credit program count based on a unique com- 
bination of CIP and program length remains complicated and may not reflect how colleges 
categorize their own programs. For example, while lowa tracks all non-credit programs by 
CIP code and reports program outcome data by different ranges of contact hours for the 
same CIP, individual CIP numbers may or may not represent “individual programs” because 
a non-credit program can consist of sets of courses reported under different CIP numbers. 


In order to shed light on patterns of enrollment and outcomes in non-credit programs, 
lowa assigns all contact hours accumulated by a student to an identified primary CIP code, 
even if the student takes a course or courses outside the CIP, and reports enrollment and 
outcomes under that primary CIP code. lowa further breaks out these data within each CIP 
code by groups of contact hours (32-99 contact hours, 100-200 contact hours, and over 200 
contact hours). This reporting structure creates reliably sized cohorts for non-credit program 
outcomes and enables comparisons between credit and non-credit programs. But as a 
result, the lowa data cannot provide any reliable proxy of the number of unique non-credit 
programs represented by a CIP code and associated award level at a single school. lowa 

is currently working to establish a data infrastructure and reporting system to identify and 
track unique programs at each school. 


SHORT-TERM PROGRAMS IN THE SHADOWS: WHAT DATA SHOW ABOUT PROGRAM LENGTH, COST, AND PAYOFF 


Student Enrollment 


No demographic data specific to short-term programs were available from lowa.*” However, the data do 
provide some insight into the general employment and education patterns of students enrolled in the 
community college system's non-credit short-term programs.*® Nine percent of students accumulating 
between 150 and 599 contact hours had earned a credit award prior to enrollment. Programs focused 
on homeland security, law enforcement, firefighting and related protective services; and computer and 
information sciences and support services have the highest shares of students with prior credit awards. 


Soil 2: Share of Students in lowa Non-Credit Programs with Prior Credit Award, by Educational 
ocus 


Homeland security, law enforcement, firefighting 
and related protective services 


Computer and information sciences 
and support services 


Education 


Engineering technologies 
and engineering-related fields 


Mechanic and repair technologies/ 
technicians 


Construction trades 


Business, management, marketing, 
and related support services 


Transportation and materials moving 


Educational Focus (2-digit CIP) 


Health professions and related programs 


Precision production 


Agriculture, agriculture operations 
and related sciences 


0% 5% 10% 15% 20% 25% 


While most students in short-term programs in the lowa Community College System enroll with no prior 
credit award, the majority of them are employed both prior to and during enrollment. 


Across all non-credit short-term program areas in the lowa data, 70 percent of students were employed 
prior to enrollment; this figure ranged from 21 percent among short-term programs focused on agricul- 
ture, agriculture operations, and related sciences to 86 percent among homeland security, law enforce- 
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ment, firefighting and related protective services programs. Three-quarters of students were employed 
during their enrollment, ranging from no students enrolled in agriculture, agriculture operations, and 
related sciences programs to 90 percent among students enrolled in programs related to engineering 
technologies and engineering-related fields. 


Program Outcomes 


Labor Market Outcomes 


lowa is one of two data sources that include some reporting on employment and earnings outcomes. 
lowa data include employment rate and median post enrollment earnings for for-credit programs, and 
employment rate and median earnings for non-credit programs. 


Students who enrolled in non-credit short-term programs identified in the lowa Community College 
data collectively have an employment rate of 90 percent one year after leaving their program, which 
is 20 percentage points higher than their collective pre-enroll ment employment rate of 70 percent. 
Programs focused on engineering technologies and engineering-related, and health professions and 
health-related fields had the highest post-enrollment employment rates (96% and 93% respectively). 


Employed students in non-credit short-term programs in lowa typically earn $27,000 in the year follow- 
ing their enrollment, an average increase of $4,350 (19%) over their pre-enrollment annual earnings.*? At 
the same time, typical post-enrollment wages range dramatically. The very few students leaving educa- 
tion-related programs had the highest earnings ($76,490 among an enrollment cohort of 13), followed 
by those leaving computer and information sciences and support service programs ($61,100 among an 
enrollment cohort of 99 students). Programs focused on health-related professions account for 44 per- 
cent of all non-credit short-term program enrollment (3,096 students), and the 93 percent of students 
employed following enrollment in these programs typically earned $17,240. 
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GRAPH 3: lowa Non-Credit Program Labor Market Outcomes by Educational Focus 
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Educational Focus (Number of Students in Enrollment Cohort) 


Average annual wage changes following enrollment range widely from 2 percent 
among transportation programs to a 44 percent increase among health-related pro- 
fessions. In several program areas with particularly high post-enrollment wages, wage 
gains were small because students earned high wages prior to enrollment. For instance, 
students who graduated from education programs earned $76,490 after enrollment 

but had pre-enrollment earnings over $70,000 already. Programs focused on education 
related fields were the only group to show a decline in the share of students employed 
after enrollment (from 77% prior to enrollment to 69% post-enrollment), and students 

in programs focused on transportation and materials moving saw the greatest gains in 
employment rates (from 49% to 89%). 


Typical post- 
enrollment wage 
gains range widely. 
In several program 
areas with 
particularly high 
post-enrollment 
wages, gains were 
small because 
students earned 
high wages prior to 
enrollment. 
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hale 4: Labor Market Outcomes for Non-Credit Short-Term Programs in lowa Community College 
ystem 


Program Focus, in Descending Order of Post-Enrollment Wages 


sXe [Ulor-hulelar-lM melee 


(2-Digit CIP) 


Post Enrollment Employment Rate 
Average Median Annual Pre-Post 


Pre-Post Employment Rate Gain 
Wage Gain 


Number of Students in Cohort* 
(percentage point increase) 
Average Median Annual Wage 


Post Enrollment** 
Percent Wage Increase 


Education 13 69% -8% $76,490 $3,670 5% 

Computer and Information Sciences 
and Support Services 99 88% 6% $61,100 $4,980 9% 

Engineering Technologies and 
Engineering-Related Fields 314 96% 14% $52,910 $2,820 6% 

Mechanic and Repair Technologies 
#lechnicians 402 92% 11% $50,990 $5,840 13% 

Homeland Security, Law Enforcement, 
Firefighting and Related Protective Services 354 90% 4% $45,120 $7,650 20% 
Construction Trades 138 91% 10% $37,440 $8,910 31% 
Precision Production 481 82% 12% $33,220 $9,660 41% 
Business, Management, Marketing, 290 70% 9% $29,960 $2,480 9% 

and Related Support Services 
Transportation and Materials Moving 1,875 89% 40% $25,940 $540 2% 
Health Professions and Related Programs 3,096 93% 14% $17,240 $5,290 44% 
Agriculture, Agriculture Operations, 29 34% 14% $7,980 $1,210 18% 

and Related Sciences 
Total 7,105 90% 20% $27,000 $4,350 19% 


*Cohort sizes refer to program enrollment. Fewer students may be included in employment and earnings cohorts. 


**Wages are rounded to the nearest $10 and are weighted by cohort size (# employed) to reflect wide ranges in number of students in different 
program cohorts. Annual wages are derived by multiplying the weighted median quarterly wages by four. 


Outcome data are available for just two of the four short-term for-credit programs that are not currently 
eligible for the Pell Grant: the Truck and Bus Driver/Commercial Vehicle Operator and Instructor and 
Emergency Medical Technology/Technician (EMT Paramedic) programs have one year post-enrollment 
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employment rates of 93 percent and 87 percent, respectively. Employed students from these programs 
typically earned $38,654 and $31,625, respectively, one year following their enrollment. 


Conclusion 


To inform legislative debates over the potential Pell eligibility of short-term programs, we set out to 
identify what state- and system-level data show about these programs, including who enrolls in them 
and how well they serve their students. Using data from three states, it was difficult to identify the num- 
ber of programs operating today that could become newly eligible for Pell Grants under legislative pro- 
posals to reduce the current minimum program length requirement and extend eligibility to non-credit 
programs. The absence of consistently collected data on existing length requirements of short-term 
programs presents a particular and considerable challenge to understanding the scope and scale of 
programs being considered for expanded Pell Grant eligibility. Moreover, other key pieces of informa- 
tion — such as student demographics, completion rates, and labor market outcomes — are also inconsis- 
tently collected across states and systems that track short-term programs, underscoring not only how 
little is known about short-term programs, but how little is knowable with the existing data. 


Nonetheless, the data show a diverse landscape of short-term programs focused on a wide range of 
fields with different program costs and a range of success rates for students. Some short-term programs 
offer students pathways to employment, some of which provide significant earnings. At the same time, 
these data demonstrate a very wide range of actual post-enrollment earnings. And while this variability 
suggests the need for some quality assurance should Pell Grant eligibility be extended to these pro- 
grams, available data are not capable of identifying consistent characteristics of programs with relatively 
high payoff. 


The Pell Grant serves as the federal government's most important investment in higher education, yet it 
has long suffered from inadequate funding, resulting in a historically low purchasing power that already 
leaves current recipients more likely to borrow student loans, and more likely to graduate with higher 
student debt.°° The grant also suffers from prior cuts to student eligibility, only some of which have 
been restored. Congress has more reason than ever to take care in protecting and strengthening this 
critical program by meeting the program's current urgent needs and ensuring it is not put at new risk of 
abuse. 


Ultimately, available outcome data specific to short-term programs tell an inconsistent story. Given that 
the current program length eligibility standards for federal aid are grounded in the history of the abuse 
of federal funds, more work is needed for institutions and states to improve their ability to accurately 
track information critical to ensuring that students are well served by these programs. 
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APPENDIX 


TABLE A: Outcomes Variables Available for Analysis by Source 


aaa eed ne 


SHORT-TERM PROI 


Number of completers 


Number 
of non-completers 


Completion rate 


On-time 
completion rate 


Number of successful 
credential attainment 
attempts 


Number of unsuccessful 
credential attainment 
attempts 


Credential 
attainment rate 


Number employed 


Employment rate 


Job placement rate 


Mean or Median 
Earnings 


Earnings gains 


Subsequent 
enrollment in a 
postsecondary or 
workforce training 
program 


Subsequent degree 
attainment 


Student loan default 


Not available 


Not available 


Not available 


Not available 


Not available 


Not available 


Available for 10 of 199 
programs 


Not available 


Not available 


Not available 


Not available 


Not available 


Not available 


Not available 


Not available 


Available for 29 of 140 
program groups 


Not available 


Not available 


Not available 


Not available 


Not available 


Not available 


Number working 4th 
quarter 


Share of completer 
cohort employed in 
4th quarter 


Not available 


4th quarter mean and 
median wage, mean 
annual wage 


Not available 


Not available 


Not available 


Not available 


Not available 


Not available 


Not available 


Not available 


Not available 


Not available 


Not available 


Non-credit programs: Number in cohort 
matched to UI records for at least one of the 
four quarters following the individual's exit 
from the last credit or non-credit course 
For-credit programs: Number in cohort not 
enrolled and matched to UI records for tax year 


Share of enrolled cohort not enrolled and 
matched to UI wage records in at least one of 
the quarters in tax year 


Not available 


Non-credit programs: Adjusted and unadjusted 
quarterly median wages for individuals in cohort 
matched to UI wages records in at least one of 
the four quarters following the individual's exit 
from the last non-credit course. 


For-credit programs: Adjusted and unadjusted 
annual median wages for graduates with wages 
covered by UI tax during that year. 


Can be calculated using reported pre- and 
post-enrollment median wages 


Number in non-credit cohort enrolled in a credit 
program in an [A community college during the 
year following enrollment in non-credit program 


Not available 


Not available 


Available Data 


Missouri 


TABLE B: Available Program Length Variables by Data Source 


Both contact or credit 
hours and weeks 
of instruction 


Required weeks or days 


Required contact 
hours for non-credit 
programs 


Required credit hours 
for credit programs 


Typical contact hours 
accumulated in 
non-credit programs 


No 


19 of 136 non-credit 
programs 


117 of 136 non-credit 
programs 


41 of 63 for-credit 
programs* 


No 


Availability varies 
by source 


Availability varies 
by source 


No 


No 


Level of precision 
varies by source 


Level of precision 
varies by source 


*Another 21 for-credit programs were identified as short-term based on the associated award type but have no associated program length. 
One for-credit program had program length measured in contact hours. 


PAGE 30 


Endnotes 


' National Skills Coalition. Fact Sheet. March 2019. https://www.nationalskillscoalition.org/resources/publications/file/Funding-Cuts-Fact- 
Sheet-March2019.pdf. 


2 The U.S. Department of Education defines a non-credit program as one in which “a student does not earn college credit and the course is 
not applicable towards a college degree.” U.S. Department of Education, Office of Career, Technical, and Adult Education, “Availability of 
Data on Noncredit Education and Postsecondary Certifications: An Analysis of Selected State-Level Data Systems.” 2014. https://files.eric. 
ed.gov/fulltext/ED555237.odf. Non-credit courses are generally not graded or recorded in a transcript, but in some cases students can acquire 
continuing education units, certification, or information to sit for a certification exam. Boeke, Marianne, and Jeannie Yockey-Fine. “State 
Authorization and Non-Credit Courses and Programs.” April 2014. httos://wcet.wiche.edu/documents/talking-points/state-auth-non-credit- 
courses. 


3 Xu, Di, and Xiaotao Ran. September 2015. “Noncredit Education in Community College: Students, Course Enrollments, and Academic Out- 
comes.” Community College Research Center. https://files.eric.ed.gov/fulltext/ED560759. pdf. 


4 Belfield, Clive and Thomas Bailey. March 2017. “The Labor Market Returns to Sub-Baccalaureate College: A Review.” Center for Analysis of 
Postsecondary Education and Employment. https://capseecenter.org/labor-market-returns-sub-baccalaureate-college-review/. 


5 We received and analyzed comparatively robust data from a fourth state, including details on program-level demographics, completion 
rates, and credential attainment rates, but removed all references to these analyses at the request of program administrators. Our analyses 
from the fourth state do not alter the conclusions presented in this paper. 


° For acomplete description of program eligibility rules, see: 2018-2019 FSA Handbook Volume 2: School Eligibility and Operations. https:// 
ifap.ed.gov/fsahandbook/attachments/1819FSAHbkVo|2Ch2. pdf. 


7 Whitman, David. February 13, 2017. “The Reagan Administration’s Campaign to Rein In Predatory For-Profit Colleges.” The Century Founda- 
tion. https://tcf.org/content/report/reagan-administrations-campaign-rein-predatory-profit-colleges/. 


8 Whitman, David. March 9, 2017. “When President George H.W. Bush ‘Cracked Down’ on Abuses at For-Profit Colleges.” The Century Foun- 
dation. https://tcf.org/content/report/president-george-h-w-bush-cracked-abuses-profit-colleges/; see also Richard W. Moore, “Proprietary 
Schools and Direct Loans,” in Selected Issues in the Federal Direct Loan Program: A Collection of Commissioned Papers, U.S. Department of 
Education, Office of Postsecondary Education, Report No. ED/OUS-94-11, 1994, p. 18. https://files.eric.ed.gov/fulltext/ED368302.pdf. 


? We received and analyzed comparatively robust data from a fourth state, including details on program-level demographics, completion 
rates, and credential attainment rates, but removed all references to these analyses at the request of program administrators. Our analyses 
from the fourth state do not alter the conclusions presented in this paper. 


10 We rely almost exclusively on the data provided; the absence of a particular data point does not mean that it might not be available from 
another source, but rather cannot be known using the available data alone. 


1 Missouri did provide some credential attainment rate data, but they included only 10 short-term programs. Due to this very small sample 
size, we do not report these data. 


12 2018-2019 FSA Handbook Volume 2: School Eligibility and Operations. February 2019. P 2-39. https://ifap.ed.gov/fsahandbook/attach- 
ments/1819FSAHbkVol2Master.pdf. 


13 Institutional awards are those requiring fewer than 15 Semester Credit Hours (SCH) or 360 continuing education contact hours for comple- 
tion of a course or series of courses that represent achievement of an identifiable skill proficiency. Occupational Skills Awards (formerly known 
as Marketable Skills Achievement Awards) require 9 to 14 semester credit hours for for-credit courses or 144-359 contact hours for workforce 
continuing education courses. See Texas Higher Education Coordinating Board. 2015. Guidelines for Instructional Programs in Workforce 
Education, p. 21, http://www.thecb.state.tx.us/reports/pdf/3378.pdf. 


4 Bureau of Labor Statistics. “Standard Occupational Classification.” https://www.bls.gov/soc/. Institutions are not required to report Occupa- 
tional Skills Awards, and data that are available are voluntarily reported and may not be complete. 


8 National Center for Education Statistics. “CIP 2010.” https://nces.ed.gov/ipeds/cipcode/Default.aspx?y=55. 


‘6 CIP codes can be mapped to SOC codes using an established data crosswalk in order to shed light on what types of occupations programs 
of a certain focus typically relate to (in other words, describing potential labor supply, but not actual expected workforce outcomes). The rela- 
tionship between the two codes allows for a single CIP code to link to multiple SOCs, which reflects the reality that a single program major or 
focus can lead to multiple types of occupations. Where we add a SOC code mapped to CIP codes included in our data, we use the 2010 SOC 
codes. 


” As defined by the U.S. Department of Education, “Credit is recognition of attendance at or performance in an instructional activity that can 
be applied toward the requirements for a postsecondary degree, certificate, or other formal award, irrespective of the activity’s unit of mea- 
surement.” https://files.eric.ed.gov/fulltext/ED555237.pdf. 


8 Research suggests that few students stack awards. See Bailey, Thomas, and Clive R. Belfield. 2017. “Stackable Credentials: Do they have la- 
bor market value?” Community College Research Center (CCRC), Working Paper No. 97. httos://ccrc.tc.columbia.edu/media/k2/attachments/ 
stackable-credentials-do-they-have-labor-market-value.pdf. 


U.S. Department of Labor, Employment & Training Administration (ETA). “Workforce Innovation & Opportunity Act: WIOA Overview.” 
https://www.doleta.gov/wioa/about/overview/. 
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0 Bird, Kisha, Marcie Foster, and Evelyn Ganzglass. September 2014. “New Opportunities to Improve Economic and Career Success for 
Low-Income Youth and Adults.” CLASP. https://www.clasp.org/sites/default/files/public/resources-and-publications/publication-1/KeyProvi- 
sionsofWIOA-Final.pdf. 


41 WIOA eligibility is not indicated one way or another for an additional 21 programs. 


22 Schneider, Mark and Rooney Columbus. October 20, 2017. “Degrees of opportunity: Lessons learned from state-level data on postsecond- 
ary earnings outcomes.” American Enterprise Institute. htto://www.aei.org/publication/degrees-of-opportunity-lessons-learned-from-state-lev- 
el-data-on-postsecondary-earnings-outcomes/; Baum, Sandy. February 2014. “Higher Education Earnings Premium: Value, Variation, 

and Trends.” The Urban Institute. httos://www.urban.org/sites/default/files/publication/22316/413033-Higher-Education-Earnings-Premi- 
um-Value-Variation-and-Trends.PDF. 


3 Individual school inventory data are available for download online http://www.thecb.state.tx.us/apps/programinventory/InvSearch.cfm. Upon 
request, we received a single report for all schools. 


4 The Texas Gainful Employment reports are a state effort, independent of the federal Gainful Employment data calculation and reporting 
requirements under the federal Gainful Employment regulation. Separate Texas GE reports are available for non-credit continuing education 
(CE) and credit (non-CE) programs. http://www.txhighereddata.org/reports/performance/ctcasalf/gainful.cfm. 


5 A continuing education course is “a Coordinating Board-approved higher education technical course offered for continuing education units 
and conducted in a competency-based format. Such a course provides a quick and flexible response to business, industry, and student needs 
for intensive preparatory, supplemental or upgrade training and education and has specific occupational and/or apprenticeship training objec- 
tives," See Texas Higher Education Coordinating Board. August 14, 2017. “Glossary of Terms.” http://www.thecb.state.tx.us/reports/PDF/1316. 
PDF. 


6 Level 1 Certificate programs ("CERT1 programs”) offered for credit must have at least 15 but no more than 42 Semester Credit Hours (SCH) 
and be designed to be completed in a year or less. See Texas Higher Education Coordinating Board. 2015. Texas Guidelines for Instructional 
Programs in Workforce Education, p. 19. http://www.thecb.state.tx.us/reports/pdf/3378.pdf. 


77 Programs related to occupations in Community and Social Service; Legal; Building and Grounds Cleaning and Maintenance; Sales and 
Related; Farming, Fishing, and Forestry; and Transportation and Material Moving each make up just 1 percent of all short-term programs in the 
Texas data. Occupational focus is not mutually exclusive because program CIP codes can match to multiple SOC codes. 


28 We use the clock:credit hour conversion of 37.5:1. For more information on this conversion, see 2018-2019 FSA Handbook Volume 2: School 
Eligibility and Operations. https://ifap.ed.qov/fsahandbook/attachments/1819FSAHbkVol2Ch2.pdf, P 2-39. 


2° Program groups are a set of programs sharing a six-digit CIP code and credit status at a unique institution. See “Challenges of Identifying 
Outcomes Specific to Short-Term Programs, Part 2” for more information. 


30 Employment rate represents the share of graduates identified as employed using UI records in Texas, which excludes out-of-state employ- 
ment and employment exempt from reporting UI wage data. 


31 For list of data available in the Texas Gainful Employment report, and their definition and methodology, see http://www.txhighereddata.org/ 
reports/performance/ctcasalf/gainfuldef.pdf. 


32 The program inventory includes a total of 3,555 unique Level 1 Certificate programs at 82 institutions. A total of 343 of these unique pro- 
grams at 70 institutions have credit hours less than 16 or contact hours less than 600; none are below 9 credit hours or 150 contact hours. Col- 
lapsing the unique programs in the inventory files into groups aligned with the GE file reduces the total program count from 3,555 programs to 
1,941 program groups. 


38 Mixed length program groups were excluded from the analysis of short-term program outcomes because over half of the programs in each 
mixed length group typically exceeded the maximum length of a short-term program, leading to a high degree of uncertainty that the pro- 
gram group's outcome data only represent short-term programs. 


34 Annual median wages are derived by annualizing (multiplying by four) the 4** quarter median wages, and rounded to nearest $10. This calcu- 
lation may differ from actual median wages, in particular for programs with small cohort sizes. Median wages include graduates working while 
enrolled in subsequent programs as well as those not subsequently enrolled (i.e. only working). 


35 Coordinating Board for Higher Education. September 12, 2018. “MoSCORES Update.” https://dhe.mo.gov/cbhe/boardbook/documents/ 
Tab28--0912.pdf. 


36 MoSCORES. “Education and Training Program Search.” https://scorecard.mo.gov/scorecard/Search. 


37 While the MoSCORES website displays state-calculated completion and earnings metrics for the mostly public programs with enrollment 
data, as well as some demographic information, Missouri had only been collecting non-credit student data since mid-2016, so most programs 
still did not display employment outcomes at the time the data were extracted for this report. Furthermore, to alleviate issues related to small 
cell sizes, e.g. few students and completers in any one program at any one institution, Missouri also uses multi-year cohorts for all public-facing 
employment and wage outcomes in MoSCORES, and results for smaller programs will take longer to become available. 


38 These programs are between 150 and 599 contact hours, four to 15 semester credit hours, or eight to 14 weeks. 
3° None of the programs identified as short-term included a length measure in months. 
“ One for-credit program had program length measured in contact hours. 


“| A total of 19 programs with length measured in weeks, and another 21 programs identified as short-term based on the associated award 
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type but missing program length, are not included in this table. We use the clock:credit hour conversion of 37.5:1. For more information on this 
conversion, see 2018-2019 FSA Handbook Volume 2: School Eligibility and Operations. httos://ifap.ed.gov/fsahandbook/attachments/1819F- 


SAHbkVoI2Ch2. pdf, P 2-39. 


* Calculations include only programs with non-missing values for the respective program licensure and credential variable. 


* Of the 199 Missouri programs, a total of 105 (53% of 199) programs have non-missing values for each of the three variables related to licen- 
sure or industry recognized credentials. Of these 105 programs, four had ‘yes’ in each of the three fields and 49 had ‘no’ in each field. 


“ Cost can be calculated for 177 of the 199 short-term programs in Missouri. All costs are rounded to the nearest $10. 


“ Community Colleges & Workforce Preparation Division of the lowa Department of Education. September 2018. lowa Community Colleges 
Employment Outcomes: Noncredit Career and Technical Education (CTE) Programs: Academic Year 2015-2016. https://educateiowa.gov/sites/ 
files/ed/documents/Noncredit%20outcomes%20report%20final%20web O.pdf. 


4 Reports on both credit and non-credit program outcomes are available online, see https://www.educateiowa.gov/iowa-community-col- 
lege-program-outcomes. To identify which among the programs in these public reports fall within the proposed program length limits for 
expanded Pell Grant eligibility, we received supplemental information from lowa Community College administrators. 


4” Publicly available outcome data for non-credit programs include age, gender, and race/ethnicity, but small cell sizes at breakouts for pro- 
grams of different lengths trigger suppression due to privacy concerns and so are not available for short-term programs specifically. 


“8 The lowa non-credit program outcomes report includes shares of students in the program cohort who had a prior degree, as well as the 
share employed in the year prior to enrollment and during enrollment. These, and labor market outcome data, are reported by three different 
program lengths (32-99 contact hours, 100-200 contact hours, and over 200 contact hours), and are reported here only for programs identified 
with separate program inventory data as being between 150 and 599 contact hours. 


“ Annual post-enrollment earnings figures are derived from quarterly unadjusted median wages during the year following program enroll- 
ment during the 2015-16 academic year. While quarterly earnings are reported at the program level in the data, quarterly median earnings and 
median earnings by educational focus summarized here have been weighted by the number of students employed to account for the wide 
ranges in program cohort sizes in the data. Quarterly earnings are annualized (multiplied by four) and rounded to the nearest $10. Cohort sizes 
included for the share employed and the median annual wages may be different, and may be less than the total enrolled. Cohort size reported 
in the accompanying graph refer to total enrolled. 


° TICAS. 2018. Pell Grants Keep College Affordable for Millions of Americans. https://ticas.org/sites/default/files/pub _files/overall_pell 
one-pager.pdf. 
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